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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). (n no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above Is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) ly^NTHS from the mailing date of this connmunication. 

• Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the OfRce later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to cx)mnnunication(s) filed on 4-16-2004 . 
2a)n This action is FINAL. 2b)S This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims , 

4) ^ Claim(s) 1-22 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-22 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement 

Application Papers 

9) n The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 27 June 2000 is/are: a)n accepted or b)K objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

1 1) n The proposed drawing correction filed on is: a)D approved b)n disapproved by the Exanniner. 

If approved, corrected drav^ngs are required in reply to this Office action. 

12) n The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)aAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
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1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) O Notice of Drattsperson's Patent Drawing Review (PTO-948) 5) Q Notice of Infomial Patent Application (PTO-152) 

3) [U Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) O Other: 
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DETAILED ACTION 



Drawings 

This application has been filed with informal drawings which are acceptable for 
examination purposes only. Formal drawings will be required when the application is 
allowed. 

Ciaim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-22 are rejected under 35 U. S. C. 103 (a) as being unpatentable over 
U.S. Patent # 5,767,842 ("Korth") in view of U.S. Patent # 6.407,679 ("Evans et a!."). 

As for claims 1, 8-10, 14, 17, and 21. Korth teaches a camera Fig.1 (2) 
corresponding to one or several cameras, a computer side of the interface there need 
only be a way to detect the fingertip motions of the keyboard operator (column 4, lines 
27-29) corresponding to one or more memories with CPU connected to the cameras, 
and processes running in the CPU that associates gesture movements with typing and 
window may appear only during the input of data (column 4, lines 54-55) corresponding 
to the produce gesture associated textual output. A virtual keyboard 3 serves as data or 
command input device. From the computer side of the interface there need only be a 
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way to detect the fingertip motions of the keyboard operator (column 4, lines 25-29) 
corresponding to sensing the typing gestures. It is inherent in the art that the gestures 
are associated with the most probable keys that would be typed if the keyboard were 
presented. Accordingly, Korth teaches all the claimed limitations as recited in claims 1, 
8-10, 14, 17, and 21 with the exception of providing a classify groups of gestures 
associate each of the classes with one or more possible keys, assign a probability to 
each of said possible keys, and integrate the probabilities assigned to the possible keys 
to identify a word for a sequence of gestures. 

However, Evans et aL teach (Fig. 1), where each key, or finger, would represent 
several different alphanumeric characters (column 2, lines 40 and 41) and a glove 
usable in a virtual environment where each finger has a group of letters associate with 
one of the keys in the virtual keyboard (column 8, lines 13-30) corresponding to classify 
each gesture into one of the plurality classes depending on the movement of the 
gestures and associate each of the classes with one or more possible keys, assign a 
probability to each of said possible keys, and integrate the probabilities assigned to the 
possible keys to identify a word for a sequence of gestures. It means that each key 
represents a classification of several alphanumeric characters. 

It would have been obvious to utilize the finger labeled as taught by Evans et al. 
in the user command system disclosed by Korth because this would allow the user 
command to match each key sequence with one or more words in the dictionary. 

As for claims 2, 1 1 , and 18-19, Korth teaches a keyboard that should provide the 
user with the key information and feedback. Tactile feedback of virtual keyboard is given 
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by the contact of the finger-tip with the working plate (column 7, lines 14-17) 
corresponding to a feedback is provided to the user on what kind of keys are associated 
with the user's gestures. 

As for claims 3 and 12, Korth teaches a visualization of the actual key function 
allows the handling of keyboards with multiple shift keys and subgroup display for large 
characters sets (column 3, lines 1-4) corresponding to displaying keys on the display, a 
keyboard for musical instalments (column 3, line 11) corresponding to playing sound 
labels for keys and playing special sound indicators, a video sensor that is used for 
monitoring the keyboard to generate image data representing positions of an operator's 
hand with respect with the keys (column 3, lines 55-58) corresponding to displaying 
image indicators on the display, a camera Fig. 1 (2) corresponding to projecting the 
keyboard to any surface and displaying picture of the keyboard with the user's hand. 

As for claim 4, Korth teaches FIG. 2 shows the area viewed by camera 2. In the 
observed region 4 the user's hands are detected. The enlarged section 5 of FIG. 2 
represents an area of the camera's image which is chosen for further image processing 
corresponding to one or more cameras, another image evaluation procedure then helps 
the image processor to match the virtual keyboard with the template. The place-mat 
may be applied on a piece of soft tissue (e.g. polyurethane foam) to reduce the strain on 
the finger-tips from the impact on a rigid desk plate (column 7, lines 47-52) 
corresponding to one or more memories with CPU connected to the cameras, and A set 
of simple intuitive 'gestures' may be defined to control the operation of a virtual 
keyboard (column 6, lines 7 and 8) corresponding to processes running in the CPU that 
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associates gesture movements with typing and produce gesture associated with textual 
output and a gesture capturing module that captures gestures through camera sensors, 
and the shape of the hands can be used to send gestures with additional information to 
the optical interface (column 2, lines 55 and 56) corresponding to an integrator module 
that integrate sequence of candidates of most probable keys into unique output key 
sequence. 

As for claim 5 and 6. Korth teaches a method of user command or data 
acquisition in a processing system that allows for analysis of human hand and finger 
motions. The activities are then interpreted as operations on a physically non-existent 
input device. The resulting optical user/computer interface complements the visual 
screen-to-user path with a hand-to-image data acquisition link from the user to the 
processor unit. It allows for the emulation of different types of manually operated input 
devices (mouse, keyboard, etc.). Mechanical input units can be replaced by virtual 
devices, optimized for the current application and for the user's physiology. The speed, 
simplicity and avoidance of ambiguity of manual data input will be maintained. Sensing 
of finger motions can generate feedback to reduce the hazard of repetitive strain injury 
(RSI) syndrome (abstract). 

As for claim 7, Korth teaches a method of data input using video sensors for 
monitoring positions of an operator's hand with respect to keys on a virtual keyboard 
optically produced on a surface. 

As for claims 1 3, and 20, Korth teaches a keyboard needs not to exist physically, 
a virtual keyboard 3 serves as data or command input device. From the computer side 
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of the interface there need only be a way to detect the fingertip motions of the keyboard 
operator. The task of the keyboard is to provide information about the available keys 
and their location, to sense the depression of the key by the operator and to produce 
some feedback virhen a key has been touched (column 4, lines 21-29). 

As for claims 15 and 22, Korth teaches a set of simple intuitive 'gestures' may be 
defined to control the operation of a virtual keyboard (column 6, lines 7 and 8) 
corresponding to means for associating gesture classes with individual typing keys. 

As for claim 16. Evans et al. teach the process loops back to select another 
word. This process continues until all the words have been placed in the data structure 
(column 4, lines 62-64) corresponding to providing training data in words or sentences 
with certain timing data. 

Response to Amendment 

Applicant's arguments filed 4-16-2004 have been fully considered but they are 
not persuasive. The applicant argued that the prior art, Evans et al., does not teach "the 
computer processes, those finger movements are transformed into text and classify the 
finger gestures into classes, and associate each of these classes with one of the keys of 
the keyboard". Examiner disagrees with the applicant because even if the prior art does 
not teach word for word the limitation claimed but the concept is inherently there. The 
prior art teaches whereby each key, or finger, would represent several different 
alphanumeric characters (column 2, lines 40 and 41) where the examiner is interpreted 
in Fig.1 as each key or finger represents different classes of gestures. And it says again 
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further down in the prior art that the left-hand ring finger, for example, could represent 
"s", "v", and "x", the right-hand index finger could represent "j", "u," "y", "h" and "n", and 
so on for each finger (column 2, lines 43-46) where the left-hand ring finger represents a 
class of gestures and the right-hand index finger represents another class of gestures. 
The applicant argued that the prior art does not teach "each word is identified on the 
basis of the probabilities assigned to the possible keys assigned to the gesture classes 
that make up that word" but examiner does not agree with the applicant because the 
prior art, Evans et al., teaches the word cat might become 228 if the letters c and a were 
both overloaded on the finger labeled key 2, and the letter t was on the finger 8 (column 
4, lines 43-46) corresponding to "each word Is identified on the basis of the probabilities 
assigned to the possible keys assigned to the gesture classes that make up that word". 
The claims have been amended but not enough to differentiate over the prior art. They 
need to be further amended to overcome the prior art of record. Therefore, the rejection 
Is maintained as was rejected in the previous office action. 

Conclusion 

Any Inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jean Lesperance whose telephone number is (703) 
308-6413. The examiner can normally be reached on from Monday to Friday between 
8:00AM and 4:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richard Hjerpe, can be reached on (703) 305-4709 . 
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Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks Washington, D.C. 20231 
or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. 
Jean Lesperance 
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